[Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) #73778

cnasikas · 2020-07-30T09:04:32Z

Summary

This PR refactors the Jira connector to be generic. Specifically:

Support of Jira alerts.
Creation of two subactions: issueTypes and fieldsByIssueType

UI:

API

Push incident:

Pushes an incident to Jira

Endpoint: api/actions/action/<action_id>/_execute
Method: POST

Payload:

{
    "params": {
        "subAction": "pushToService",
        "subActionParams": {
            "savedObjectId": "66ce8cf1-2c0a-4d9a-9030-bff20d6c4690",
            "title": "A title",
            "description": "A description",
            "priority": "Low",
            "issueType": "10006",
            "labels": ["kibana", "alerts"],
            "comments": [
                {
                    "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                    "comment": "A comment"
                },
                {
                    "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                    "comment": "Another comment"
                }
            ]
        }
    }
}

Response

{
    "status": "ok",
    "data": {
        "title": "RJ2-8",
        "id": "10202",
        "pushedDate": "2020-09-01T10:58:55.157Z",
        "url": "https://siem-kibana.atlassian.net/browse/RJ2-8",
        "comments": [
            {
                "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                "pushedDate": "2020-09-01T10:58:56.453Z"
            },
            {
                "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                "pushedDate": "2020-09-01T10:58:56.977Z"
            }
        ]
    },
    "actionId": "bda54773-aadf-4668-aead-69a72b5e83b4"
}

Get issue types:

Get issue types of a specific project (projectKey is configured at the creation of the connector).

Endpoint: api/actions/action/<action_id>/_execute
Method: POST

Payload:

{
    "params": {
        "subAction": "issueTypes",
        "subActionParams": {}
    }
}

Response

{
    "status": "ok",
    "data": [
        {
            "id": "10023",
            "name": "Bug"
        },
        {
            "id": "10024",
            "name": "Improvement"
        },
        {
            "id": "10017",
            "name": "Story"
        },
        {
            "id": "10008",
            "name": "My custom issue"
        }
    ],
    "actionId": "bda54773-aadf-4668-aead-69a72b5e83b4"
}

Get fields of issue type:

Get the fields of a specific issue type.

Endpoint: api/actions/action/<action_id>/_execute
Method: POST

Payload:

{
    "params": {
        "subAction": "fieldsByIssueType",
        "subActionParams": {
            "id": "10024"
        }
    }
}

Response

{
    "status": "ok",
    "data": {
        "summary": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "issuetype": {
            "allowedValues": [
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/issuetype/10024",
                    "id": "10024",
                    "description": "An improvement or enhancement to an existing feature or task.",
                    "iconUrl": "https://siem-kibana.atlassian.net/secure/viewavatar?size=medium&avatarId=10310&avatarType=issuetype",
                    "name": "Improvement",
                    "subtask": false,
                    "avatarId": 10310
                }
            ],
            "defaultValue": {}
        },
        "attachment": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "duedate": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "description": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "project": {
            "allowedValues": [
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/project/10015",
                    "id": "10015",
                    "key": "RJ2",
                    "name": "RJ2",
                    "projectTypeKey": "business",
                    "simplified": false,
                    "avatarUrls": {
                        "48x48": "https://siem-kibana.atlassian.net/secure/projectavatar?pid=10015&avatarId=10412",
                        "24x24": "https://siem-kibana.atlassian.net/secure/projectavatar?size=small&s=small&pid=10015&avatarId=10412",
                        "16x16": "https://siem-kibana.atlassian.net/secure/projectavatar?size=xsmall&s=xsmall&pid=10015&avatarId=10412",
                        "32x32": "https://siem-kibana.atlassian.net/secure/projectavatar?size=medium&s=medium&pid=10015&avatarId=10412"
                    }
                }
            ],
            "defaultValue": {}
        },
        "assignee": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "priority": {
            "allowedValues": [
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/1",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/highest.svg",
                    "name": "Highest",
                    "id": "1"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/2",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/high.svg",
                    "name": "High",
                    "id": "2"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/3",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/medium.svg",
                    "name": "Medium",
                    "id": "3"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/4",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/low.svg",
                    "name": "Low",
                    "id": "4"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/5",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/lowest.svg",
                    "name": "Lowest",
                    "id": "5"
                }
            ],
            "defaultValue": {
                "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/3",
                "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/medium.svg",
                "name": "Medium",
                "id": "3"
            }
        },
        "labels": {
            "allowedValues": [],
            "defaultValue": {}
        }
    },
    "actionId": "bda54773-aadf-4668-aead-69a72b5e83b4"
}

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser
This was checked for cross-browser compatibility

For maintainers

This was checked for breaking API changes and was labeled appropriately

Resolves #56426

elasticmachine · 2020-08-31T15:11:39Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

elasticmachine · 2020-08-31T15:11:40Z

Pinging @elastic/siem (Team:SIEM)

pmuellr

Looking good! I've noted some things that I think will need to be changed, mainly regarding schema.maybe() usage and the rename of the config field.

Note, I haven't looked at the UI code yet, figured I'd post what I have so far before waiting to review that.

x-pack/plugins/actions/server/builtin_action_types/jira/api.ts

x-pack/plugins/actions/server/builtin_action_types/jira/schema.ts

x-pack/plugins/actions/server/builtin_action_types/jira/case_schema.ts

x-pack/plugins/actions/server/builtin_action_types/jira/schema.ts

x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts

x-pack/plugins/case/server/routes/api/cases/configure/get_connectors.ts

pmuellr · 2020-09-01T19:16:16Z

x-pack/plugins/actions/README.md

+| Property              | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | Type                |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- |
+| apiUrl                | ServiceNow instance URL.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | string              |
+| incidentConfiguration | Case configuration object. The object should contain an attribute called `mapping`. A `mapping` is an array of objects. Each mapping object should be of the form `{ source: string, target: string, actionType: string }`. `source` is the Case field. `target` is the ServiceNow field where `source` will be mapped to. `actionType` can be one of `nothing`, `overwrite` or `append`. For example the `{ source: 'title', target: 'short_description', actionType: 'overwrite' }` record, inside mapping array, means that the title of a case will be mapped to the short description of an incident in ServiceNow and will be overwrite on each update. | object _(optional)_ |


Seems like you are renaming an existing config value of casesConfiguration to incidentConfiguration. If you do that, any existing actions with casesConfiguration will be broken - they won't validate anymore.

I don't believe we had to deal with this yet, but I think the way to do it would be like the way we treat kibana.yml config values that we "rename". Basically, you need to deprecate the old one, and not delete it. In code that checks the value, you will potentially have to check both (use the new one unless it's not set, otherwise use the old one). @mikecote other thought?

I'm not sure if we would support some way for these to be migrated.

Good catch @pmuellr, I saw the same rename got done in ServiceNow too. From what I see, the impact is that the connector would "lose" the field mappings because incidentConfiguration would be empty for existing ServiceNow connectors (possible bug in Cases app?) and the user would have to re-configure the mappings. I'm not sure how it would validate when updating 🤔

One way I see this being fixed since we're not GA yet is with a saved object migration in the actions plugin that moves the value from casesConfiguration to incidentConfiguration (including decrypt / encrypt process). In a post GA world, @pmuellr is correct and we'd probably have to think of a non-breaking way to support both until the next major where possible.

I'm indifferent what approach is done (migrate or revert to casesConfiguration or make users re-configure mappings). Since there's a phase 2 to remove this value from connectors, we could always skip it until the field gets removed.

@pmuellr @mikecote I followed ServiceNow PR example. As mapping for the moment is fixed to a default one (only thought the UI), creating or updating a case or validating the mapping will not be a problem. What Cases will loose is the appended text (Example: A title in Jira (created at 2020-04-22T18:18:03.896Z by Christos Nasikas)) in titles and descriptions. I totally agree with what you are saying but I think it's ok if some users lose temporarily that functionality as in phase 2 we gonna append this information beforehand in Cases. I think the easiest one is to revert back to casesConfiguration and lose that functionality only for ServiceNow. @XavierM What are your thoughts on that?

As of today, this mapping is static meaning nobody can change it. So we can always default back but I think it will be better to do a migration. @cnasikas I can show you how to do it if you do not know, but this one seems pretty simple.

We could deal with a migration today, since the action is in the actions plugin, but in the future when we have actions NOT in the actions plugin, it's not clear to me if they can or should be doing migrations against the action SO's. Basically, in the future, we may need to have a per-actionType migration story that could be used outside of the actions plugin. Actually makes me wonder if every actionType (and alertType) should be their own saved object, or something, to make lifecycle management like this a little more straight-forward.

pmuellr · 2020-09-01T19:22:31Z

...test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira.ts

@@ -123,7 +126,8 @@ export default function jiraTest({ getService }: FtrProviderContext) {
          config: {
            apiUrl: jiraSimulatorURL,
            projectKey: mockJira.config.projectKey,
-            casesConfiguration: mockJira.config.casesConfiguration,
+            incidentConfiguration: mockJira.config.incidentConfiguration,


regarding the rename of casesConfiguration - since actions created with the old config value could already exist from the previous release, should probably have a test for that. Might be hard to test for. Especially hard since the field is part of AAD so you can't just overwrite the field with the generic saved object API. Probably a case for es-archiver, sorry to say. another one that @mikecote may have some input on

++ es archiver is probably best in this scenario but there may be issues where it erases other data the test suite relies on (users / spaces). @gmmorris is going through the exact same issue in the RBAC work, maybe a change he's working on will help here?

I think based on my note above: #73778 (comment). I'd be +1 on reverting it back to casesConfiguration since we know the field is going away soon and we don't have to worry about a migration or tests to cover this.

x-pack/test/alerting_api_integration/basic/tests/actions/builtin_action_types/jira.ts

…ne) (elastic#73778)

…hase one) (#73778) (#77049)

* master: (68 commits) a11y tests on spaces home page including feature control (elastic#76515) [ML] Transforms list: persist pagination through refresh interval (elastic#76786) [ML] Replace all use of date_histogram interval with fixed_interval (elastic#76876) [Timelion] Update timelion deprecation links (elastic#77008) [Security Solution] Refactor Network Details to use Search Strategy (elastic#76928) Upgrade elastic charts to 21.1.2 (elastic#76939) [Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) (elastic#73778) [Snapshot & Restore] fix pre existing policy with no existing repository (elastic#76861) Update saved object management UI text (elastic#76826) [Form lib] Add validations prop to UseArray and expose "moveItem" handler (elastic#76949) [Logs UI] Use fields api in log stream (elastic#76919) [UI Metrics] Support multi-colon keys (elastic#76913) [APM] Script for creating functional test archive (elastic#76926) [ENDPOINT] First version of the trusted apps list. (elastic#76304) Correct field for rum page url (elastic#76916) [Security Solution] Fix redirect properly old SIEM App routes (elastic#76868) Bump http-proxy from 1.17.0 to 1.18.1 (elastic#76924) [RUM Dashboard] Visitor breakdown usability (elastic#76834) [Search] Add a new advanced setting searchTimeout (elastic#75728) [DOCS] Adds timelion deprecation to new visualize docs structure (elastic#76959) ...

kibanamachine · 2020-09-09T21:43:51Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 627a3f2

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
securitySolution	1967	-5	1972
triggers_actions_ui	241	+17	224
total		+12

async chunks size

id	value	diff	baseline
securitySolution	9.9MB	-31.8KB	10.0MB
triggers_actions_ui	894.5KB	+157.1KB	737.3KB
total		+125.3KB

miscellaneous assets size

id	value	diff	baseline
securitySolution	326.0KB	-27.7KB	353.7KB
triggers_actions_ui	27.7KB	+27.7KB	0.0B
total		-0.0B

page load bundle size

id	value	diff	baseline
securitySolution	810.0KB	-6.5KB	816.6KB
triggers_actions_ui	274.6KB	+10.5KB	264.1KB
total		+3.9KB

distributable file count

id	value	diff	baseline
default	45464	+5	45459

History

💚 Build #73340 succeeded 627a3f2
💚 Build #72898 succeeded 11f824b6ae2e8642f75e6aa3d91bd8693df691bf
💚 Build #72851 succeeded 6b1b32b78b3418b636e7764b2c00d9023a276e42
💚 Build #72796 succeeded 6a23270fc6b89488850954fb8c2991518e0b766c
💔 Build #72720 failed a3d60c9bd6763ea81326f7268139df61519264a5

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cnasikas · 2020-10-19T08:30:28Z

@gchaps

API changes for creating a Jira connector:

casesConfiguration renamed to incidentConfiguration
Added optional attributeisCaseOwned.

elasticmachine · 2021-09-23T14:28:05Z

Pinging @elastic/security-solution (Team: SecuritySolution)

cnasikas self-assigned this Jul 30, 2020

cnasikas added release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Team:SIEM v7.10.0 v8.0.0 labels Jul 30, 2020

cnasikas force-pushed the refactor_jira_phase_one branch 5 times, most recently from 1750196 to a003995 Compare August 4, 2020 14:31

mikecote mentioned this pull request Aug 11, 2020

JIRA Action type #56426

Closed

cnasikas force-pushed the refactor_jira_phase_one branch 5 times, most recently from 904c152 to 4f1f865 Compare August 30, 2020 15:42

cnasikas changed the title ~~[Alerting][Connectors][skip-ci] Refactor Jira: Generic Implementation (phase one)~~ [Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) Aug 30, 2020

cnasikas mentioned this pull request Aug 31, 2020

[Security Solution] [Threat Hunting] [Cases] Allow User to Specify Atlassian Jira Ticket fields from Cases UI #76212

Closed

cnasikas force-pushed the refactor_jira_phase_one branch from 4f1f865 to 1342bf5 Compare August 31, 2020 15:01

cnasikas marked this pull request as ready for review August 31, 2020 15:11

cnasikas requested review from a team as code owners August 31, 2020 15:11

cnasikas force-pushed the refactor_jira_phase_one branch from 72a1f83 to a3eb0ef Compare September 1, 2020 10:47

YulNaumenko self-requested a review September 1, 2020 16:14

pmuellr self-requested a review September 1, 2020 16:31

pmuellr requested changes Sep 1, 2020

View reviewed changes

cnasikas added 9 commits September 9, 2020 13:35

Improve README

6b8339a

Move common types and schema to case folder

1a43d83

Remove findIncidents function

c9d843b

Refactor for inclusive language

97a67c2

Make actionConnector optional

f2723d9

Move case mappings to its own folder

ccd74cb

Fix i18n

e8c3a25

Improve UX when editing an alert

caabc6e

Improve imports

627a3f2

cnasikas force-pushed the refactor_jira_phase_one branch from 11f824b to 627a3f2 Compare September 9, 2020 10:49

XavierM approved these changes Sep 9, 2020

View reviewed changes

cnasikas merged commit db5652e into elastic:master Sep 9, 2020

cnasikas deleted the refactor_jira_phase_one branch September 9, 2020 13:17

cnasikas mentioned this pull request Sep 9, 2020

[7.x] [Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) (#73778) #77049

Merged

cnasikas added a commit to cnasikas/kibana that referenced this pull request Sep 9, 2020

[Alerting][Connectors] Refactor Jira: Generic Implementation (phase o…

53dcb7f

…ne) (elastic#73778)

cnasikas added a commit that referenced this pull request Sep 9, 2020

[7.x] [Alerting][Connectors] Refactor Jira: Generic Implementation (p…

6302386

…hase one) (#73778) (#77049)

cnasikas mentioned this pull request Sep 21, 2020

[Actions] Migrate cases configuration object #77998

Merged

2 tasks

This was referenced Sep 29, 2020

[Actions][Jira] Set parent issue for Sub-task issue type #78772

Merged

[Actions] Jira and IBM Resilient tests #79060

Closed

[SIEM][CASE] Add generics to connector's types #64506

Closed

cnasikas mentioned this pull request Oct 15, 2020

[Security Solution][Cases] Fix bug with case connectors #80642

Merged

2 tasks

cnasikas added release_note:breaking and removed release_note:skip Skip the PR/issue when compiling release notes labels Oct 19, 2020

cnasikas added release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. and removed release_note:breaking labels Oct 21, 2020

MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) #73778

[Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) #73778

cnasikas commented Jul 30, 2020 •

edited by mikecote

Loading

elasticmachine commented Aug 31, 2020

elasticmachine commented Aug 31, 2020

pmuellr left a comment

pmuellr Sep 1, 2020

mikecote Sep 1, 2020

cnasikas Sep 2, 2020 •

edited

Loading

XavierM Sep 2, 2020

pmuellr Sep 3, 2020

pmuellr Sep 1, 2020

mikecote Sep 1, 2020

mikecote Sep 1, 2020

kibanamachine commented Sep 9, 2020

cnasikas commented Oct 19, 2020

elasticmachine commented Sep 23, 2021

[Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) #73778

[Alerting][Connectors] Refactor Jira: Generic Implementation (phase one) #73778

Conversation

cnasikas commented Jul 30, 2020 • edited by mikecote Loading

Summary

API

Push incident:

Get issue types:

Get fields of issue type:

Checklist

For maintainers

elasticmachine commented Aug 31, 2020

elasticmachine commented Aug 31, 2020

pmuellr left a comment

Choose a reason for hiding this comment

pmuellr Sep 1, 2020

Choose a reason for hiding this comment

mikecote Sep 1, 2020

Choose a reason for hiding this comment

cnasikas Sep 2, 2020 • edited Loading

Choose a reason for hiding this comment

XavierM Sep 2, 2020

Choose a reason for hiding this comment

pmuellr Sep 3, 2020

Choose a reason for hiding this comment

pmuellr Sep 1, 2020

Choose a reason for hiding this comment

mikecote Sep 1, 2020

Choose a reason for hiding this comment

mikecote Sep 1, 2020

Choose a reason for hiding this comment

kibanamachine commented Sep 9, 2020

💚 Build Succeeded

Build metrics

@kbn/optimizer bundle module count

async chunks size

miscellaneous assets size

page load bundle size

distributable file count

History

cnasikas commented Oct 19, 2020

elasticmachine commented Sep 23, 2021

cnasikas commented Jul 30, 2020 •

edited by mikecote

Loading

cnasikas Sep 2, 2020 •

edited

Loading